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Abstract. There are limited research 
studies about the development of test 
instrument to assess the level of entrepre- 
neurial thinking among children in STEM 
education. The purpose of this research 
was to develop an Entrepreneurial Science 
Thinking Test (ESTT) for primary school 
children in STEM Education and evaluate 
its validity and reliability. The ESTT was de- 
veloped using experiential learning theory 
which comprised of five constructs, namely 
Observation, New Ideas, Innovation, 
Creativity, and Value. The ESTT consisted of 
ten open-ended question items that require 
children to answer questions in statements 
and draw sketches of ideas. The evaluation 
was conducted to determine the reliability 
and validity of ESTT which involved five 
subject matter experts and 166 11-year- 
old fifth graders from five urban schools in 
Sabah, Malaysia. The data obtained from 
fifth graders were computed using WIN- 
STEPS software version 3.73 and analysed 
using the Rasch measurement model. The 
results indicated a high acceptable content 
validity and construct validity, high internal 
consistency, and excellent item reliability 
and item separation. Through item fit 
analysis, all items were retained. The find- 
ing established the reliability and validity 
of the ESTT and would therefore represent 
avalid and highly reliable instrument for 
measuring entrepreneurial science thinking 
among fifth graders in STEM Education. 
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Introduction 


Entrepreneurial thinking (ET) is one of many critical skills that 21st 
century learners require in facing an increasingly competitive world. ET has 
the potential to enrich human capital in gaining innovative knowledge, 
new revolutionary ideas, and strong ethical standards (Bacigalupo et al., 
2016). ET can be defined as a cognitive situation that seeks creative and 
innovative ideas as well as opportunities (Krueger, 2005). ET is not an ability 
that needs to be mastered in order to become a mere entrepreneur, but 
it is a soft skill that is required to enhance human growth, meet the job 
market and improve productivity (Bacigalupo et al., 2016). In producing 
learners who can master the characteristics and ethics of entrepreneurship, 
handle existing resources well and face future challenges efficiently, this ET 
is very important (Edwards-Schachter et al., 2015). Learners can be more 
analytical, creative, and inventive thinkers, successful in communication, 
and ethical workers with these skills (Lekashvili, 2013). 

The importance of ET for supplying skilled workers who can com- 
municate different soft skills in Science, Technology, Engineering and 
Mathematics (STEM) fields has been acknowledged by some countries 
in their 21% century curricula (Borowczak, 2015). However, integrating ET 
is still not widespread in STEM education. Eltanahy et al. (2020) stressed 
the importance of applying ET in STEM learning to raise the awareness of 
STEM design among learners and inspire them to be more entrepreneurial. 
Learners studying STEM without ET skills are less likely to have the capabil- 
ity to determine the value of products derived from the socio-economic 
aspects of society. This is significant because learners are currently showing 
high trends (61.4%) towards STEM-based careers (Hatisaru, 2021). Swartz 
and McGuinness (2014) stressed that thinking skills need to be explicitly 
taught in curricula, while Gunawan & Shieh (2020) recommended that ET 
is integrated into any STEM curriculum to produce a significant effect on 
learning. 
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Realizing the need for entrepreneurial thinking in STEM, the importance of ET began to be examined by 
some Malaysian local scholars. Buang et al. (2009) have proposed a combination of science process skills and 
entrepreneurial thinking known as Entrepreneurial Science Thinking (EST). This concept integrates problem- 
solving ideas and entrepreneurial elements to train learners to search for thorough ideas in solving a problem 
creatively (Syukri et al., 2013b). However, there is little research on the development of test instruments to 
assess the level of entrepreneurial science thinking among primary school children in STEM education. It is 
therefore important to develop an ET test for STEM education to ensure that ET is nurtured in STEM education 
from primary school level (Liu & Zhi, 2010; Menzies, 2012; Sheehan et al., 2018). The Entrepreneurship Science 
Thinking Test (ESTT) was developed in line with this to assess entrepreneurial science thinking among fifth 
graders in STEM education. 


Research Problem 


Entrepreneurial science thinking begins with an important basic science skill which is the skill of creating 
(Daniel, 2016; Neck & Greene, 2011). In the concept of classroom assessment in Malaysian schools, learners 
with good creating skills can achieve the mastery level six (ML6). However, based on the Classroom Assess- 
ment Mastery Achievement Report (Tawau District Education Office, 2019), the number of tenth graders who 
achieved ML6 for Science subjects was only 4%, far behind when compared to ML5 (23.82%) and ML4 (44.25%). 

Furthermore, based on the PISA Report 2018 (Schleicher, 2019), the average score of Malaysian ninth 
graders in terms of science literacy is only 438, below the international average score of 489. This is a discon- 
certing scenario because the nation holds high aspirations to achieve a nation of creators by 2030. Therefore, 
it is obvious that entrepreneurial science thinking needs to be infused from the primary school level to better 
develop the children’s science literacy, critical thinking, and reasoning skills (Hoachlander & Yanofsky, 2011; 
National Research Council, 2011). 

The application of entrepreneurial science thinking in STEM Education is now considered significant in 
preparing children today to venture into STEM-related careers (Dabney et al., 2012; Sadler et al., 2012; Wyss et 
al., 2012). Such application will encourage children to solve challenging and meaningful daily problems as well 
as improve their cognitive reasoning skills (Hunter et al., 2016). In addition, entrepreneurial science thinking 
opens up space for learners to think more broadly to explore new ideas in STEM-based problems (English et 
al., 2017). Children with entrepreneurial science thinking are able to learn STEM in real context and develop 
their STEM literacy in order to succeed in the modern economic era (Tsupros et al., 2009). In fact, they are also 
able to face the challenges of daily life related to the field of STEM (Bybee, 2013). In addition, entrepreneurial 
science thinking also encourages children to improve Science literacy (McDonald, 2016) and engineering de- 
sign skills emphasized in STEM (Afriana et al., 2016; Jin & Bierma, 2013; Kennedy & Odell, 2014; Kuenzi, 2011; 
Zollman, 2012). It is clear that the application of entrepreneurial science thinking in STEM Education needs to 
be implemented explicitly starting at a lower school level to achieve the country’s desire to produce STEM- 
skilled human capital capable of solving global problems, making decisions, and innovating creations for the 
benefit of future societies. 

Although entrepreneurial thinking has been introduced in the Malaysian curriculum, the development of 
instruments that measure the level of entrepreneurial science thinking among primary school children in STEM 
education is not yet widespread (Buang et al., 2009; Syukri et al., 2013a). Instruments that have been developed 
previously only measure the readiness of the integration of entrepreneurial science thinking (Ishak et al., 2014) 
and teacher pedagogical knowledge in teaching entrepreneurial science thinking (Syukri et al., 2013b). 

Li et al. (2016) developed an instrument for measuring entrepreneurial thinking among engineering stu- 
dents. The instrument was developed based on Kern Entrepreneurial Engineering Network (KEEN) framework 
that is specialised in the context of engineering. The instrument focuses on three aspects, namely Curiosity, 
Connections, and Creating Value. The instrument reduced its items from 37 items to 29 items after the validity 
and reliability analysis using exploratory factor analysis. Unfortunately, the items do not reflect the development 
of new ideas and positive values in students’ creation. Bolton and Lane (2012) developed an Individual Entre- 
preneurial Orientation (IEO) instrument for testing entrepreneurial thinking among 1,100 university students. 
By using exploratory factor analysis, the instrument was found reliable and valid in assessing three of the five 
dimensions: innovativeness, risk-taking, and proactiveness. However, the measurements are comparatively 
restricted in these three dimensions. The extent to which the students’ observation of current materials and 
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designs, as well as the contribution of inventions to society, was not assessed. Schelfhout et al. (2016) con- 
structed an instrument to assess entrepreneurial competence using a behavioural indicator scale, in addition 
to measuring entrepreneurial thinking. The instrument was developed with 11 sub-competencies that assess 
high school students’ competence and entrepreneurial thinking. 

To sum up, most instruments constructed for measuring entrepreneurial thinking are appropriate for 
learners of higher education and secondary school. There are few and limited instruments targeting entrepre- 
neurial science thinking for primary school children in STEM education. Hence, an instrument for assessing 
entrepreneurial science thinking for STEM Education needs to be developed and introduced at a lower school 
level. In line with this, the Entrepreneurship Science Thinking Test (ESTT) was developed, and its validity and 
reliability were assessed to ensure its usability in measuring entrepreneurial science thinking among fifth 
graders in STEM education. 


Literature Review 
Science Entrepreneurial Thinking 


The concept of Entrepreneurial Science Thinking (EST) is a notion of science teaching and learning to pro- 
duce learners who are equipped with entrepreneurial thinking (Syukri et al., 2013a). EST is referred to as design 
thinking skills based on scientific knowledge and entrepreneurial orientation (Buang et al., 2009). Buang et 
al. (2009) proposed the Entrepreneurial Science Thinking Model (ESTM) which provided the theoretical model 
of entrepreneurial thinking in this study. ESTM consisted of five constructs namely Observation, New Ideas, 
Innovation, Creativity, and Value as described below. 


Observation 


The observation construct refers to the activity of observing various phenomena found in the environment 
that is related to the concept of science (Buang et al., 2009). The observation construct, as described in this 
study, is the activity of making observations in a planned and purposeful manner. Learners make observations 
about the appropriate materials and designs used to make a new product. 


New Ideas 


There is a need for new ideas through phenomena that have been observed in the form of ideas, systems, 
models, designs, or products (Buang et al., 2009). In this research, the new ideas construct suggests that learn- 
ers generate ideas by continuously thinking about the use of materials and the design of materials used in 
new products to seek uniqueness. 


Innovation 


Buang et al. (2009) stressed that the innovation construct refers to the activity of learners selecting some 
ideas that can be improved and evaluating the selected ideas. In this research, the innovation construct allows 
students to select ideas that can be modified or improved from the previous steps. In the future, the selec- 
tion of these ideas for change helps in creating product designs. Learners then need to evaluate the ideas by 
specifying the reasons for the idea they selected. 


Creativity 


The definition of creativity can be applied in a concentrated way as the task of creating new ideas (Buang 
et al., 2009). In this study, the creativity construct refers to the task of reinforcing and evolving ideas in a con- 
centrated way. Learners express their new ideas by sketching designs, labelling, and building product design 
models. Learners are then asked to state the name of the product, the price offered and the target group of 
buyers for the product produced. 
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Value constructs refer to activities to ensure that the ideas or products produced are beneficial to society. 
In terms of cost savings, product function as well as values and ethics in product invention, learners need to 
state the benefits of their products. This is to cultivate values such as love of the environment and the practice 
of using sustainable materials in the production of products for the community. Learners then present their 
model in front of the class and present the benefits of their products. 


Experiential Learning Theory in ESTT Development 


Kolb (1984)'s experiential learning theory defines learning as “the process whereby knowledge is created 
through the transformation of experience” (p. 41). Kolb proposed that learners learn through a circle of ex- 
periences that lead to observations and reflections. These reflections are applied to prior understanding and 
converted into abstract concepts or theories, resulting in forms and behaviours to adapt to new experiences 
that can be tested and explored. As claimed by Gemmell and Kolb (2013), experiential learning theory is in 
line with the concept of entrepreneurial science thinking which is based on learning through reflection and 
experience. Entrepreneurial science thinking needs to go through social and cognitive processes where learners 
develop new ideas and transform those ideas so that they can be used in the market and society (Gemmell et 
al., 2012). Thus, this theory provided the theoretical framework of ESTT development where learners use their 
experience to solve consumer problems and produce innovative products and services for the benefit of society. 

Kolb (1984) further reiterated that individuals progress through the four stages of the learning cycle that 
are interconnected, namely concrete experience, reflective observation, abstract conception, and active ex- 
perimentation (Figure 1). During the stage of concrete experience, learners are involved with new experiences 
which become the basis of observation. Next, in reflective observation, individuals reflect on their observation 
and begin to construct a general theory of what this information means. In the next step of abstract conceptu- 
alization, learners organize information systematically and logically into concepts, theories, and ideas. Finally, 
during the active experimentation stage, learners test the implications of these concepts in new situations. At 
this stage, learners are able to see things from various perspectives and act creatively and innovatively. The 
process once again returns to the first stage of the experience process. 

The learning process of Kolb (1984)’'s theory is in line with ESTT development, which encourages learners 
to experience the phases of Observation, New Ideas, Innovation, Creativity, and Values. Learners were exposed 
to issues through concrete experiences that would lead to the emergence of various ideas, which is in line 
with Observation in ESTT. This process leads to reflective thinking where learners obtain ideas and inspiration 
to invent products that could help in problem-solving. Hence, the reflective thinking stage is in line with New 
Ideas in ESTT. In the abstract conceptual stage, learners began to organize ideas and translate ideas in the 
form of prototype sketches. This is where innovation and creativity in ESTT take place. At this stage, through 
the experience they went through, learners invent their prototype. Carlsson et al. (1976) explained that the 
process of implementing solutions to issues would be carried out during the stage of active experimentation. 
Accordingly, learners evaluated their products by ensuring that the products invented could solve issues and 
benefit the users. As explained by Krakauer et al. (2017), the stage of active experimentation is the stage where 
learners relate their perceptions to the actual context. In this study, the actual context refers to the context of 
society where learners applied their STEM knowledge and experience in inventing the products that would 
benefit society. This stage represents the Value construct of ESTT. 
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Learning cycle in Kolb’s Experiential Learning Theory (1984) 


Concrete 
Experience 
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Abstract 
Conceptualization 


Rasch Measurement Model 


The Rasch Measurement Model (RMM) is an efficient solution for the development of in-depth statistics to 
provide high validity and reliability of ESTT instrument (Bond & Fox, 2015). RMM analyses each respondent's ability 
to respond to the instrument and measures the difficulty of each item in the instrument (Wolins et al., 1982). RMM 
is also able to assess latent characteristics, such as human thoughts and emotions (Aziz et al., 2015). 

The RMM built on the basis of Item Response Theory is among the adequate statistical models as it can cal- 
culate the item's difficulty and the individual's ability to be evaluated at the same time (Deane et al., 2016). As a 
result, the RMM was able to classify the items and person validity and reliability. Furthermore, Rasch analysis can be 
used in terms of item polarity, item and respondent misfit, as well as dimensionality, to perform construct validity. 

While Rasch analysis can take a longer process than conventional analysis, Rasch analysis may offer a deeper 
understanding of the instrument's strengths and shortcomings (Boone & Scantlebury, 2005). Bond and Fox (2007) 
claimed that RMM, by statistical analysis, is an efficient solution for developing highly valid and reliable instru- 
ments. The researchers used Rasch analysis to assess the validity and reliability of the ESTT instrument based on 
these specified strengths. 


Research Focus 
The main focus of this research was to develop an instrument on entrepreneurial science thinking in STEM 
Education for 11-year-old children in primary schools. Validity and reliability are two important factors to consider 
when developing an instrument. Hence for the need to develop a valid and reliable tool, Rasch analysis was used 
to examine the validity and reliability of ESTT. 
Research Aim and Questions 
This research was aimed to develop a valid and reliable instrument for assessing entrepreneurial science 


thinking in STEM Education among fifth graders using the Rasch Measurement Model. There were three research 
questions guiding this research: 
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Q1: IsESTT feasible in assessing entrepreneurial science thinking among fifth graders in STEM Educa- 
tion? 

Q2: How valid is ESTT in assessing fifth graders’ science entrepreneurial thinking in STEM Education 
based on: 1) Person fit statistics; 2) Item fit statistics; 3) ltem Polarity; and 4) Confirmation of uni- 
dimensionality? 

Q3: Howreliable is ESTT in assessing the entrepreneurial science thinking in STEM Education among 
fifth graders according to: 1) KR-20 Cronbach's coefficient alpha; 2) Person and item reliability 
indices? 


Research Methodology 
Research Design 


A survey design using a test instrument was employed to obtain data on fifth graders’ entrepreneurship 
thinking level. In education, a survey research design is well established as it can analyse topics and constructs 
efficiently and economically (Creswell & Creswell, 2017). The data gathered from the ESTT instrument was focused 
on the fifth graders’ experiences of integrating the entrepreneurship elements in science project-based learning. 
This research was carried out between November and December 2019. 


Research Sample 


The ESTT was distributed to 166 samples that were chosen randomly from five primary schools in the Tawau 
district, Sabah, Malaysia. A sample size with the range of 108-243 is wide enough to have 99% confidence that 
the item difficulty can be measured within + 1/2 logit of its stable value (Linacre, 1994). The research sample had a 
similar background where the selected schools were grouped in urban school clusters. The sample consisted of 87 
girls (52.4%) and 79 boys (47.6%) aged between 10 to 11 years. Approximately 60% of parents were government 
employees, while 40% were working in the private and industry sectors. 


Ethical Considerations 


Written consent was obtained from the parents and school principals prior to the administration of ESTT. 
Initially, an outline of the intent of the ESTT which was to measure the level of entrepreneurial science thinking 
was meted out. The consent letter detailed the fifth graders’ involvement in the research and the parents’ consent 
reflecting their understanding of the research purpose. All the fifth graders were assured of the confidentiality of 
their responses and complete anonymity. Fifth graders were also informed that anyone could withdraw from the 
research without penalty. A follow-up briefing session was then carried out to explain the guidelines and proce- 
dures for answering questions in ESTT. 


Administration of ESTT 


Each fifth grader was asked to read the instructions carefully before answering the ESTT according to their respec- 
tive knowledge without the help of other children. Fifth graders relate current information, insights, and experiences 
to the scientific principles and entrepreneurial aspects surrounding them throughout answering questions, such as 
building materials, the use of construction materials, design, uniqueness of an invention, and the impact of an inven- 
tion on society. Fifth graders were given an hour to think and articulate their responses. Open questions were given 
according to the construct and fifth graders were required to answer the questions in the order of the constructs. The 
answers were collected and reviewed in advance to ensure the fifth graders followed the correct instructions and 
provided complete answers before the data are analysed using WINSTEPS software version 3.73. 


Instrumentation 


Entrepreneurial science thinking is the ability to create and make improvements to a product, idea, or process 
so that the product has added value from the social and economic aspects. In this regard, the relationship of sci- 
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ence concepts acquired in the classroom and the ability of teachers in nurturing EST triggers innovation to adapt 
to situations that occur and are needed in the daily life of children (Venuvinod & Sun, 2002). This concept was used 
when developing the question items in the Entrepreneurial Science Thinking Test (ESTT). 

Open-ended questions were used in measuring the entrepreneurial science thinking of each individual. 
The use of open-ended questions can help in obtaining variations in respondents’ ideas (Chen et al., 2020), provide 
an overview of respondents’ level of knowledge (Clarke & Holt, 2019), and help researchers identify misconceptions 
among respondents (Schuetz, 2010). Researchers constructed open-ended questions based on the five constructs 
in EST (Buang et al., 2009) which are Observation, New Ideas, Innovation, Creativity, and Value. 

The ESTT consisted of ten open-ended question items that require fifth graders to answer questions in state- 
ments and draw sketches of ideas. Question items were developed by referring to the content of Standard Docu- 
ments of Curriculum and Assessment (DSKP) for Year Five Science under the themes of Physical Science as well as 
Technology and Sustainable Life (Curriculum Development Division, 2019, p. 55-68). The main question asked fifth 
graders to produce cell phone designs for the use of the future community. The context of cell phone usage was 
chosen because it is one of the criteria in DSKP under the theme of Physical Science. Fifth graders were then given 
a stimulus picture and ten question items. Items were arranged according to the group of constructs to enable 
fifth graders to organize their answers which then lead them to Entrepreneurial Science Thinking. Table 1 shows 
the constructs of the Entrepreneurial Science Thinking Test (ESTT) for STEM Education. 


Table 1 
Items for ESTT by Construct 


Construct Definition Item No. Item statement 
Observation Make observations in a planned la State the materials used to make cell phones. 
and purposeful manner. 1b State the design of a cell phone that you can observe. 
New Ideas Generate ideas by looking for 2A Explain the advantages of using the materials you mentioned to build the 
uniqueness cell phone. 
2b Explain the advantages of using the cell phone design you mentioned. 
Innovation Select ideas that can be im- 3a Based on the ideas you have stated in question 2, select three (3) ideas 
proved and evaluate those ideas that you can improve to produce future cell phones. 
3b Why did you choose the above ideas? Give three (3) reasons. 
Creativity Strengthen and improve ideas in Reinforce and improve the three (3) ideas you have chosen for the crea- 
a focused way 4a tion of new cell phones for future community use. Sketch and label new 
features on your model in the space provided. 
4b Complete details about your product by stating the product name, price 
offered and target group of buyers. 
Value Ensure that the ideas or prod- ba State the benefits of your product to the community in terms of cost sav- 


ucts produced are beneficial to ings and product functionality. 
the community 


State the benefits of your product to the community in terms of values and 
ethics in product creation. 


5b 

Children were required to make observations in a prepared and purposeful way in the Observation Construct. 

The first item required the children to state the materials used to make a cell phone, whilst the second item required 
the children to state the cell phone design. Children were required to produce ideas relevant to individuality in 
the creation of New Ideas. By specifying the benefits of utilising the products and design requirements in the 
observation stage, children had to search for uniqueness or benefits. The fifth item in the Innovation Construct 
instructed children to choose three ideas that could be augmented and updated in order to create potential cell 
phones, while the sixth item requested children to state the reasons for choosing the ideas mentioned. The Creativ- 
ity Construct allows children to validate and strengthen concepts in a focused way. The seventh item centred on 
children strengthening and improving the concepts chosen for the development of new cell phones. They were 
requested in this item to draw and mark new features on the model they made. Next by specifying the name of the 
product, the price available, and the target category of purchasers, the eighth item advised children to complete 
the details of the product they had made. Finally, the Value Construct was where children wanted to ensure that 
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the products would help the society. The ninth and tenth items allowed children to state the advantages in terms 
of cost savings and include terms of values and ethics in product development, respectively. 

The time allocation for answering all the items was 60 minutes. The scoring rubric for the Entrepreneurial Sci- 
ence Thinking Test construct was adapted from Ho et al. (2013). There is a minimum score of 0 anda maximum score 
of three for each item given in the test. Based on the child’s response, each score was rated from 0 to 3 points. An 
example of a scoring rubric according to children’s response for one of the constructs in ESTT is shown in Table 2. 


Table 2 
An Example of the Scoring Rubric according to Children’ Response in ESTT 


Construct Ability Scoring rubric sh elsand 

cores 

3. Innovation 3.2 Make an evaluation of the ideas 3.2.4 Children can provide three (3) factors in the Level 4 

(Select ideas that can be that have been selected. selection of ideas. 3 marks 
improved and evaluate ; : ; 

those ideas) 3.2.3 Children can give two (2) factors in the selec- Level 3 

tion of ideas. 2 marks 

3.2.2 Children can provide one (1) factor in the Level 2 

selection of ideas. 1 mark 

3.2.1 Children cannot give one (1) factor in the Level 1 

selection of ideas. 0 mark 

cea uiaeaieaei: e ate ae 3.1.4 Children can choose three (3) ideas vee 

proved to produce new inventions. 3 marks 

3.1.3 Children can choose two (2) ideas one 

2 marks 

3.1.2 Children can choose one (1) idea Eevee 

1 mark 

3.1.1 Children cannot choose one (1) idea Fevel 

0 mark 


Source: Ho et al. (2013) 


Data Analysis 


The data of the research were analysed to determine the content and construct validity. To determine the value 
of content validity agreement, the researchers used the Content Validation Index (CVI). CVI provides an average 
rating of scores for all items evaluated by an expert. A CVI value can be computed for each item on a scale (I-CVI) 
as well as for the overall scale (S-CVI). Davis (1992) pointed out that for a newly designed instrument, the usual CVI 
value obtained is .80. For cases with content validation requiring three or more experts, Polit et al. (2017) recom- 
mended a rating of .78 and above for I-CVI and a minimum S-CVI of .80 for the averaging approach. 


Item Content Validation Index (I-CVI) = Total of experts in agreement 
Total of experts 


Scale Content Validation Index/Average (S-CVI/Ave) = Total I-CVI for each item on the scale 
Total of items 


For assessing construct validity and item reliability, WINSTEPS software version 3.73 was used. This en- 
sured the instrument's quality and the accuracy of the data obtained by the researcher before the instrument was 
used in actual research. The first analysis was conducted via a person fit analysis which was based on the values 
of ‘MEASURE; MNSQ Outfit and ZSTD Outfit (Edwards & Alcock, 2010). Nevin et al. (2015) asserted that if the ZSTD 
Outfit value exceeds 2.0 and the MEASURE value is high, there is a probability that excellent learners would not 
carefully answer the easy items. If the ZSTD Outfit value exceeds 2.0 but the MEASURE value is low, it is likely that 
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low-ability learners would be able to answer difficult items correctly. Therefore, unfit respondents will be eliminated 
to increase the validity of the instrument (Lamoureux et al. 2008). 

For the item fit analysis, Boone et al. (2014) as well as Bond and Fox (2015) proposed three criteria namely, Outfit 
Mean Square Values (MNSQ), Outfit Z-Standardised Values (ZSTD) and Point Measure Correlation (PTMEA-CORR). 
The MNSQ Outfit value informs the researcher about the item fit in the measurement while the PTMEA-CORR value 
indicates whether the development of the construct has achieved its goal (Bond & Fox, 2007). In another aspect, 
ZSTD provides information to the researcher whether the data obtained really conforms to the instrument model. 
Any item that fails to meet one of the criteria in Table 3 needs to be modified or dropped so that the item fit value 
can be increased (Sumintono & Widhiarso, 2015). 


Table 3 
Fit Indices for Item Fit 


Statistics Fit Indices 
Outfit mean square values (MNSQ) 0.50 - 1.50 
Outfit z-standardised values (ZSTD) -2.00 - 2.00 
Point Measure Correlation (PTMEA-CORR) 0.40 - 0.85 


Source: Boone et al. (2014) 


Rasch analysis can also be used to identify item polarity through PTMEA-CORR values. A positive PTMEA- 
CORR value indicates that the item can accurately measure what it needs to measure and vice versa if its value 
is negative. The researchers also assess the instruments’ unidimensionality to guarantee that the instrument can 
successfully measure the construct of entrepreneurial science thinking (Sumintono &Widhiarso, 2015). Component 
Analysis provides dimensional criteria based on the‘raw variance explained by measures’ (Sumintono & Widhiarso, 
2015). The accepted value of ‘raw variance explained by measures’ should exceed 20%, is considered good if it is 
more than 40% and considered excellent if more than 60%. Meanwhile, the value of ‘unexplained variance in first 
contrast’ should not exceed 15%. 

In terms of reliability, the researcher referred to Sumintono and Widhiarso (2015) for Cronbach's alpha value 
(KR-20), item-person reliability and separation indices (Table 4). The separation index of persons was used to classify 
the level of learners. A good separation index should be >2, where the higher the separation index, the better the 
level of classification of persons. Item separation index was also used to validate the item hierarchy. The low item 
separation index value, <3 shows that the sample of learners was not large enough to confirm the hierarchy of 
item difficulty in the instrument. Linacre (2002) insisted that a high separation value indicates that the instrument 
has good quality since it can identify the group of items and respondents. 


Table 4 
Reliability Measured via the Rasch Analysis 


Statistics Fit Indices Interpretation 
Cronbach's alpha (KR-20) <5 Low 
<.6 Moderate 
6-7 Good 
7-8 High 
9-1.0 Very High 
Item and Person Reliability Index <.67 Low 
.67 - .80 Sufficient 
81 -.90 Good 
91 - .94 Very Good 
>.94 Excellent 
Item Separation Index >3 Good 
Person Separation Index >2 Good 


Source: Sumintono & Widhiarso (2015) and Linacre (2002) 
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Research Results 
Content Validity 


Content validity indicates the extent to which an item adequately represents the content of a trait that a 
researcher wants to measure (Creswell & Creswell, 2017). Kline (2005) stated that an experts’ review is necessary 
to ensure the accuracy of the construct as well as the clarity of its contents. Mullen (2003) remarked that a group 
of experts are those who are trained in a specific field. Therefore, the researchers showed the ESTT to five experts 
who had vast knowledge in the fields of entrepreneurial science thinking, Science Education and STEM Education. 
The researchers used an item evaluation form adapted from the Malaysian Examinations Board (2013). The panel of 
experts evaluated ESTT items from the aspects of conformity, accuracy, and clarity as well as suitability. Comments 
from the experts were recorded and taken into consideration for the ESTT instrument improvement process. Table 
5 shows the list of experts involved in the content validation panel. 


Table 5 
Content Validation Panel for ESTT 


Name Institute Designation Expertise 

ExpertA University Professor (PhD) Science Entrepreneurial Thinking 
Teachers Training Academic Lecturer : . ‘ 

Expert B Institute (TT!) STEM Department (PhD) Curriculum and Instructional (Science) 
Teachers Training Academic Lecturer : 2 

Exper’ Institute (TTI) STEM Department mapner EalieY 

Expert D Primary School aoe science Fan STEM Education and Scientific Creativity 

Expert E Primary School Science Subject Coach Science Education (Primary School) 


Based on Table 6, all items in ESTT were accepted and the value obtained from the Scale Content Validity 
Index (S-CVI) was .92. This S-CVI value meets the requirement of = .80 as set by Polit et al. (2017) for new instru- 
ments. The S-CVI index of .92 indicated that the expert panel regarded the content validity of the ESTT instrument 
as very high and acceptable. 


Table 6 
Content Validity Index (CVI) Results of the ESTT Instrument 


Item ExpertA ExpertB ExpertC ExpertD  ExpertE priate I-CVI Results 
la - / / / / 4 80 Accepted 
1b - / / / / 4 80 Accepted 
2a - / / / / 4 80 Accepted 
2b - / / / / 4 80 Accepted 
3a / / / / / 5 1.00 Accepted 
3b / / / / / 5 1.00 Accepted 
4a / / / / / 5 1.00 Accepted 
4b / / / / / 5 1.00 Accepted 
5a / / / / / 5 1.00 Accepted 
5b / / / / / 5 1.00 Accepted 

Scale Content Validation Index/ Average (S-CVI/Ave) 92 
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Person Fit 


Table 7 displays the persons (which are the fifth graders in this case), whose responses were most ill-fitting with 
the Rasch analysis; or in other words, their responses contrasted from the estimation given by the Rasch model. The 
fifth graders were ranked according to the highest ZSTD Outfit value. Among the fifth graders, there were seven 
respondents (063, 100, 127, 166, 004, 002 and 022) who scored an Outfit ZSTD value higher than 2.0. The remain- 
ing fifth graders scored an Outfit ZSTD value within the acceptable range (from -2.0 to +2.0). Fifth graders whose 
responses gave negative PTMEA-CORR readings showed that they made decisions out of the usual. 


Table 7 
Misfit Order of the Persons in ESTT Instrument 


Parson MNSQ Outfit ZSTD Outfit PTMEA-CORR 
(.50-1.50) (-2.0-2.0) (.40 - .85) 
063 9.90 3.9 - 38 
100 9.90 4.5 - 44 
127 9.90 3.1 -.25 
166 7.33 2.5 - 12 
004 5.72 2.8 - 02 
002 2.97 2.2 08 
022 2.97 2.2 08 


In addition, there were 10 fifth graders who attained extreme marks (maximum marks) namely, students 
020, 044, 068, 076, 084, 088, 093, 096, 109, and 161 (Figure 2). A total of 17 fifth graders were excluded and 149 
respondents out of the 166 respondents were involved with the next analysis. This denotes that in the pilot study, 
the items were fit for almost all the students (89.76%) and the analysis conducted on those fifth graders showed 
quality findings for the assessment using the Rasch analysis. 


Figure 2 
Person with Extreme Scores 


INPUT: 166 Person 10 Item REPORTED: 166 Person 10 Item 3 CATS WINSTEPS 3.73 


Person: REAL SEP.: 2.44 REL.: .86 ... Item: REAL SEP.: 6.86 REL.: .98 


Person STATISTICS: MEASURE ORDER 


[ENTRY TOTAL TOTAL MODEL| INFIT | OUTFIT |PT-MEASURE |EXACT MATCH] | 
— SCORE COUNT MEASURE 5S.E. |MNSQ ZSTD|MNSQ ZSTD/CORR. EXP.| OBS% EXP%| Person| 
------------------------------------ +--- ---------+-----------+------- 
| 20 30 10 7.51 : MAXIMUM MEASURE| § .00 .00/100.0 100.0] 020 | 
| 44 30 10 Face : MAXIMUM MEASURE| § .00 .00/100.0 100.0] 044 | 
| 68 30 10 4654 i MAXIMUM MEASURE| § .00 .00/100.0 100.0] 068 | 
| 76 30 10 7< 5a a. MAXIMUM MEASURE| § .00 -00/100.0 100.0! 076 | 
| 84 30 10 F es S 1. MAXIMUM MEASURE| § .00 .00/100.0 100.0] 084 | 
| 88 30 10 f. 52 1. MAXIMUM MEASURE| § .00 .00/100.0 100.0] 088 | 
| 93 30 10 7.54 i. MAXIMUM MEASURE| ff .00 -00/100.0 100.0] 093 | 
| 96 30 10 Pe | 5 MAXIMUM MEASURE| § .00 -00/100.0 100.0] 096 | 
| 109 30 10 7-52: : MAXIMUM MEASURE| § .00 .00/100.0 100.0] 119 | 
| 161 30 10 7.52 i. MAXIMUM MEASURE .00 -00/100.0 100.0] 161 | 
| : | 29 10 6.13 : : 219 : 22 29} 90.0 90.0] O01 | 
| 3 29 10 6.13 1. -4| a || 24 29| 90.0 90.0] 003 | 
| a2 29 10 6.13 : 23 3] 22 29} 90.0 90.0] 012 | 
| 16 29 10 6.13 a -4| -1| 28 29} 90.0 90.0] 016 | 
I 19 29 10 6.13 1.1211.18 oS! a9 Pe || 22 .291 90.0 90.0! 019 | 
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Linacre (2007) emphasised that the value of MNSQ Outfit provides a strong value in determining the suit- 
ability of items for the measurement of a construct. Boone et al. (2014) ascertained that the suitable product value 
range is between .5 to 1.5 based on MNSQ Outfit, between -2 to +2 range for ZSTD Outfit and between .4 to .85 
range for PTMEA-CORR. Any items that are outside the range of Outfit MNSQ, Outfit ZSTD, and PTMEA-CORR are 
considered inappropriate (Boone et al., 2014). However, if the item meets one of the criteria, the item must be 
retained (Sumintono & Widhiarso, 2015). 

Based on Table 8, there are four items (1, 4, 7 and 9) that are outside the range of items. Boone et al. (2014) 
and Aziz et al. (2014) stated that items situated outside the range and which do not meet all three criteria are 
considered unsuitable. However, if the item meets one of the criteria, the item must be retained (Sumintono & 
Widhiarso, 2015). Table 8 shows that all the items meet at least one criterion. Thus, no items were changed and 
removed from the instrument. 


Table 8 
Misfit Order of the Items in ESTT 


Kern Outfit MNSQ Outfit ZSTD PTMEA-CORR Result 
(.50-1.50) (-2.0 - 2.0) (.40 - .85) 
I7 1.57 3.2 54 retained 
12 1.35 a 18 retained 
15 1.33 9 19 retained 
13 1.26 1.2 AT retained 
18 55 -4 16 retained 
19 36 - 9 69 retained 
16 92 -5 64 retained 
110 87 -1.0 85 retained 
14 67 -27 84 retained 
1 52 3.5 .90 retained 
Item Polarity 


Based on Table 8, the minimum value of PTMEA-CORR is .54 while the maximum value is .90. These positive 
values indicated that all the items functioned in a parallel direction while negative values showed which items 
needed to be repaired or dropped. The item polarity analysis using PTMEA-CORR values elucidates that the items 
in ESTT move in the same direction according to the measured construct (Linacre, 2002). 


Unidimensionality 


Unidimensionality is important to determine that the instrument being developed can measure in one direction 
and ensure precise results from the study. Unidimensionality is frequently detected by using the Principal Compo- 
nent Analysis of Rasch Residual (PCAR). Based on Figure 3, the observed value of the Raw Variance Explained by 
Measures is 66.8%. Values exceeding the 60% index are at an excellent level and it is proven that ESTT instruments 
have a strong unidimensionality. This shows that ESTT can measure constructs in science entrepreneurial think- 
ing. The value of 6.6% was detected from the Unexplained Variance in the 1% Contrast, which was less than 15%. 
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Figure 3 


Principal Component Analysis of Rasch Residual (PCAR) 


INPUT: 149 Person 10 Item REPORTED: 149 Person 10 Item 3 CATS WINSTEPS 3.73 


Table of STANDARDIZED RESIDUAL variance (in Eigenvalue units) 

-- Empirical -- Modeled 
30.1 100.0% 100. 0% 
20.1 ais = 65.7% 


Total raw variance in observations 
Raw variance explained by measures 


Raw Variance explained by persons = 44.9% 
Raw Variance explained by items = 6.4 21. 2% 20. 8% 
Raw unexplained variance (total) = 10.0 00.0% 34.3% 
UnexpIned variance in 1st contrast = 2.0 [eves] 29: 7% 
UnexpIned variance in 2nd contrast = 1.5 0% 15.0% 
UnexpIned variance in 3rd contrast = 1.2 4.0% 12.1% 
UnexpIned variance in 4th contrast = 1.2 3.8% 11.6% 
UnexpIned variance in 5th contrast = 1.1 3.7% 11.0% 


Reliability and Separation Indices 


In relevance to the reliability of ESTT, the person measure reliability indicates how well the fifth graders can 
be distinguished based on their responses. The item measure reliability specifies how well the items (statements) 
can be discriminated from one another based on their authenticity approved by the respondents. The reliability 
findings overview is summarized in Table 9. 

Table 9 presents the ESTT’s reliability values based on Cronbach's coefficient alpha (KR-20). The value achieved 
was .91. This infers that the reliability of the ESTT is within a very high range (Sumintono & Widhiarso, 2015). The 
item reliability value of .98 is within a very high range (Bond & Fox, 2007; Linacre, 2007). The item separation index 
was 7.97, which is more than 3.0. This value proved that ESTT has a good spread of items (Linacre, 2007) that it 
could distinguish the respondents according to the hierarchy of item difficulty in ESTT. 

The person reliability of .89 was also within a good range, whereas the person separation value in ESTT was 
2.83. Bond and Fox (2007) stated that the respondents reliability value, which is higher than .80, indicates that the 
respondents gave good and consistent feedback. The item separation value of more than 2.00 indicates there is 
enough spread of items (Linacre, 2002). The value of 2.83 points out that the fifth graders could be divided into 
three groups according to their responses to the items in ESTT. 


Table 9 
Summary of the Reliability Findings 


iced ESTT Interpretation 
Cronbach’s alpha (KR-20) 9-1.0 91 Very high 
Item Reliability >.94 98 Excellent 
Item Separation Index >3.0 7.97 Good 
Person Reliability 81 - .90 89 Good 
Person Separation Index >2.0 2.83 Good 


Discussion 


In this research, ESTT was developed to assess the level of entrepreneurial science thinking among fifth grad- 
ers. In addition to its importance in determining the extent of entrepreneurial science thinking, this ESTT also offers 
added value in the analysis of entrepreneurial science thinking in STEM Education. 

Overall, the Rasch Measurement Model was used to assess the validity and reliability of ESTT. Five constructs, 
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namely Observation, New Ideas, Innovation, Creativity and Value, were established as the focus of the ESTT in- 
strument. All ten open-ended items went through the content validation phase by five experts. The items for the 
Observation Construct were adjusted from observation about energy and the transformation of energy to the 
observation of building materials and design. This is to guarantee that the scientific principles of materials are 
tested during the Observation Construct. In addition, the researcher also changed the question's keyword from 
‘what’ to ‘explain’ for the construction of the New Ideas construct. It is to ensure that fifth graders are subjected to 
the engagement of high-level thinking. 

The construct validity analysis was conducted for item and person fit, item polarity as well as unidimensional- 
ity. The results of Rasch’s analysis show that ESTT has good psychometric quality. A positive PTMEA-CORR analysis 
indicates that all items move in the same direction in measuring the constructs to be measured (Bond & Fox, 2015; 
Linacre, 2012). Meanwhile, the obtained Raw Variance Explained by Measures value of the ESTT instrument proved 
that the ESTT instrument truly measures the construct of entrepreneurial science thinking. In other words, there 
was no sixth construct in ESTT (Aziz et al., 2015; Fisher, 2007). 

The ESTT instrument was also tested for its reliability and separation index. According to Cohen and Swedlik 
(2018), a good set of test items can be distinguished by respondents. The ESTT instrument was found to have very 
high Cronbach's alpha value, excellent item reliability value and good person reliability. These findings show that 
the reliability of ESTT instrument is high in assessing the entrepreneurial science thinking of fifth graders in STEM 
education. The obtained good item separation value shows that ESTT instrument can be categorized into eight 
strata of items levels while the obtained person separation value proves that fifth graders can be divided into three 
strata according to ability level. 

Most instruments developed by previous researchers (Li et al., 2016; Bolton & Lane, 2012; Schelfhout et al., 
2016) concentrate on high school and undergraduate students’ level of entrepreneurial thinking in the field of 
engineering and entrepreneurship education. This limitation was addressed with the development of ESTT. 

Research findings confirm that ESTT is explicitly in line with Kolb (1984)’s experiential learning theory and 
offer a new instrument for primary school children in STEM education. ESTT helps educators obtain preliminary 
data on the extent of children’s entrepreneurial science thinking in primary schools. The data can then be used to 
develop appropriate education programs to enhance entrepreneurial science thinking. As reported by Gray (2016), 
the level of entrepreneurial thinking is essential to anticipate learners’ future potential in STEM. 

This study has also shown that the level of entrepreneurial thinking among primary school children can be 
assessed by referring to the appropriate entrepreneurial Thinking Model. Moreover, ESTT utilises an open-ended 
questions approach, which allows children to offer their views more freely based on their current experience and 
knowledge (Lifan & Chen, 2009). In addition to that, most previous entrepreneurial thinking instruments used the 
exploratory factor analysis method, while ESTT uses Rasch analysis, which leads to the accuracy of the instrument 
developed. 


Conclusions and Implications 


This research has several implications that impact practicality and methodology. In terms of practicality, the 
ESTT is a new instrument developed by researchers based on Kolb’s Experiential Learning Theory and the Entrepre- 
neurial Science Thinking Model and was adapted to suit the context of STEM Education. Thus, the Entrepreneurial 
Science Thinking Test (ESTT) instrument is fit to fill the gap in STEM education research and overcome the issue of 
the lack of instruments that measure entrepreneurial science thinking among primary school children. The develop- 
ment of this instruments a significant endeavour in warranting the continuity of the application of entrepreneurial 
science thinking in STEM Education which will ultimately contribute to a generation of inventors and creators. 

In terms of methodology, the Rasch Model of Measurement is more specific and detailed in determining the 
validity and reliability of the ESTT instrument. The Rasch measurement model's analysis of instrument validity in 
terms of item-person fit, item polarity and unidimensionality showed that ESTT instrument has a high validity in 
measuring entrepreneurial science thinking in STEM education. The high reliability analysis of ESTT instrument 
with good item-person separation value proved that ESTT instrument is reliable for measuring entrepreneurial 
science thinking in STEM education for fifth graders. This study also demonstrated that instruments that have 
gone through a phase of validity and reliability will help the researcher measure the variables studied and make 
accurate decisions based on the analysis of the findings. In fact, these findings provide a basis for other research- 
ers so that the ESTT instrument can be distributed to learners in other areas. Ultimately, the validity and reliability 
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analysis using the Rasch Measurement Model successfully proved that the ESTT instrument is very suitable for real 
field study in assessing entrepreneurial science thinking in STEM education among children in primary schools. 

Even though the findings suggest that ESTT is a reliable and valid instrument for STEM education, its limita- 
tions should be acknowledged. ESTT was tested in five primary urban schools using a sample of 166 fifth graders, it 
may not be representative of the general population of primary school students. Future research needs to involve 
a bigger sample size, including rural schools. ESTT can be infused into any STEM curriculum integrated model to 
promote better critical and inventive thinking skills in STEM. It is suggested to extend the usage of ESTT instrument 
to other regions of the country, as well as to a variety of learners for greater generalisability. 
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